Collation device

ABSTRACT

A collation device includes a light source unit; a camera unit that receives light emitted from the light source unit and reflected in a collation area of an object to acquire a photographed image of the collation area; a display unit; and a processor configured to, by executing a program: display a guide of a bright spot portion of the light source unit on the photographed image on the display unit so that an angle formed by the light source unit, the camera unit, and the collation area of the object is a predetermined angle.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is based on and claims priority under 35 USC 119 fromJapanese Patent Application No. 2021-155853 filed Sep. 24, 2021.

BACKGROUND (i) Technical Field

The present invention relates to a collation device.

(ii) Related Art

JP6119751B discloses a photographing assisting tool that assists inphotographing a predetermined area of a surface of an object having fineunevenness and strong specular reflection, and includes a light sourceunit that emits light, and a cover portion that has a shape that coversthe predetermined area on the surface of the object, and in which a partof a surface corresponding to a predetermined angle range from a normaldirection facing the predetermined area is a black background and asurface corresponding to the other angle range is a light source areasurface that diffuses light emitted from the light source unit, in whichthe cover portion has an upper surface portion facing the surface of theobject and a side surface portion connected to the upper surface portionbetween the surface of the object and the upper surface portion, thelight source unit is provided on an upper surface portion side of theside surface portion of the cover portion, and the black background areaof the cover portion is provided on an inner surface of the uppersurface portion based on the predetermined angle range centered on anormal line of the surface of the object.

JP2021-47448A discloses a photographing system that acquires aphotographed image in which the same feature point can be stablyextracted by using a camera of a mobile terminal and includes anacquisition unit that acquires an image including a pattern randomlyformed on an object surface and a control unit that superimposes anddisplays a transparent image of the image acquired by the acquisitionunit on a live view image displayed on a display unit in a case where acollation object is photographed by the camera unit.

JP2020-43490A discloses a technology that can determine a reading areaand a light irradiation direction in a case of acquiring surfaceinformation of an inspection object in a case of acquiring the surfaceinformation of the inspection object and collating the surfaceinformation with surface information registered in advance. A first markand a second mark are printed and formed at predetermined positions ofthe inspection object. The first mark indicates the reading area, andthe second mark indicates the light irradiation direction. It isdisclosed that the reading area and the light irradiation direction arematched between the time of registration and the time of collation ofthe surface information by using the first mark and the second mark.

SUMMARY

In a case where a user photographs a collation area by using a mobileterminal and collates the image with a registered image, for example, itis desirable that an angle formed by a light source unit of a mobileterminal, a camera unit, and the collation area at the time ofphotographing is an appropriate angle with respect to an angle in a casewhere the registered image is acquired.

Aspects of non-limiting embodiments of the present disclosure relate toa collation device that provides a technology that allows a user toeasily recognize whether or not an angle formed by a light source unit,a camera unit, and a collation area is appropriate in a case wherephotographing the collation area.

Aspects of certain non-limiting embodiments of the present disclosureaddress the above advantages and/or other advantages not describedabove. However, aspects of the non-limiting embodiments are not requiredto address the advantages described above, and aspects of thenon-limiting embodiments of the present disclosure may not addressadvantages described above.

According to an aspect of the present disclosure, there is provided acollation device including a light source unit; a camera unit thatreceives light emitted from the light source unit and reflected in acollation area of an object to acquire a photographed image of thecollation area; a display unit; and a processor configured to, byexecuting a program: display a guide of a bright spot portion of thelight source unit on the photographed image on the display unit so thatan angle formed by the light source unit, the camera unit, and thecollation area of the object is a predetermined angle.

BRIEF DESCRIPTION OF THE DRAWINGS

Exemplary embodiment(s) of the present invention will be described indetail based on the following figures, wherein:

FIG. 1 is a system configuration diagram of an exemplary embodiment;

FIG. 2 is a configuration block diagram of a collation imagephotographing machine of an exemplary embodiment;

FIGS. 3A and 3B are plan diagrams of an object including a hologramportion and an object of paper of an exemplary embodiment;

FIG. 4 is an explanatory diagram showing a relative positionalrelationship between a QR code (registered trademark) and an ink portionof an exemplary embodiment;

FIGS. 5A and 5B are explanatory diagrams showing a positionalrelationship between a light source unit, a camera unit, and an objectof an exemplary embodiment;

FIG. 6 is an explanatory diagram of a guide showing a light sourceposition displayed in a preview image and a guide showing an ink portionof an exemplary embodiment;

FIG. 7 is an explanatory diagram showing a positional relationshipbetween a collation image photographing machine (smartphone) and anobject of an exemplary embodiment;

FIG. 8 is an explanatory diagram of various guides displayed on apreview image of an exemplary embodiment;

FIG. 9 is an explanatory diagram of a color development around an inkportion displayed on a preview image of an exemplary embodiment;

FIG. 10A is an explanatory diagram (No. 1) of a guide of an elevationangle displayed on a preview image of an exemplary embodiment;

FIG. 10B is an explanatory diagram (No. 2) of a guide of an elevationangle displayed on a preview image of an exemplary embodiment;

FIG. 10C is an explanatory diagram (No. 3) of a guide of an elevationangle displayed on a preview image of an exemplary embodiment;

FIGS. 11A to 11C are explanatory diagrams of a color development changearound an ink portion of an exemplary embodiment;

FIG. 12 is a processing flowchart of an exemplary embodiment;

FIG. 13 is a detailed flowchart of shape extraction processing of anexemplary embodiment;

FIG. 14 is an explanatory diagram of a photographing direction change ofa modified example;

FIG. 15 is a processing flowchart of a modified example;

FIG. 16 is an explanatory diagram (No. 1) of a preview image transitionof a modified example; and

FIG. 17 is an explanatory diagram (No. 2) of a preview image transitionof a modified example.

DETAILED DESCRIPTION

Hereinafter, an exemplary embodiment of the present invention will bedescribed with reference to the drawings. An individual identificationsystem that uniquely identifies an object by photographing a surfaceimage of the object and performing image collation between a registeredimage and a collation image will be described as an example.

The individual identification system has a technology that registers animage of a part of a surface of an object, specifically about 0.1 toseveral mm in advance as information unique to the object and uniquelyidentifies that the object collated is the same as a registered object,that is, the object is genuine, and the information unique to the objectis, for example, a random pattern by a fine pattern. A satin pattern isa specific example of the random pattern by the fine pattern. The satinpattern is not limited to one obtained by surface treatment such asfrosted glass, and has a concept that includes not only the satinpattern applied by treatment processing to metal, synthetic resin(plastic or the like), or the like, but also a wrinkle pattern obtainedby embossing treatment and randomly woven fiber pattern, a random finedot pattern by printing, a random particle distribution by printing withink containing glitter particles, and unevenness formed on sandblastedglass surface or the like. Further, the satin pattern includes not onlyan unintentionally formed satin pattern but also an intentionally formedsatin pattern for identification or collation. In short, the satinpattern is a random pattern that is difficult to control and form. Itmay be said to be a kind of “artifact metrics” that optically reads sucha random pattern and uses the random pattern as information.

Here, a case is assumed in which a printing substrate having unevennesssuch as a hologram and paper is used as a printing substrate and an inkportion in which metal particles are dispersed is printed on theprinting substrate having such unevenness to form a random pattern.

FIG. 1 shows a system configuration of the present exemplary embodiment.The collation system includes a registered image photographing machine20, a collation image photographing machine 22, and a server computer50. The registered image photographing machine 20 and the servercomputer 50, and the collation image photographing machine 22 and theserver computer 50 are connected to each other by a communicationnetwork.

An object 10 is irradiated with a light source unit 21 such as an LED,and the light reflected from the object 10 is photographed by theregistered image photographing machine 20 to acquire a registered image.The registered image photographing machine 20 and the light source unit22 a can be composed of dedicated equipment for registration.

An irradiation angle φ of the irradiation light from the light sourceunit 21 is set to a certain fixed angle. The acquired registered imageis transmitted to the server computer 50 and stored in a registeredimage DB 50 b in the server computer 50.

On the other hand, the object 10 is photographed by using a mobileterminal such as a smartphone held by the user of the collation systemas the collation image photographing machine 22. The object 10 isirradiated with a light source unit 22 a such as an LED mounted on asmartphone or the like, and the light reflected from the object 10 isphotographed by the camera unit 22 b mounted on the smartphone or thelike. An irradiation angle φ of the irradiation light from the lightsource unit 22 a is set to be substantially the same as an angle φ whichis the condition in a case where the registered image is acquired. Thereason is that, as described above, the random pattern of the inkportion changes depending on an irradiation direction of light, so thatit is necessary to set a positional relationship between the lightsource unit 22 a, the camera unit 22 b, and the object 10 to besubstantially the same as a positional relationship at the time ofphotographing the registered image 16.

A processor of the collation image photographing machine 22 performsseries of processing on the photographed image to extract the inkportion from the photographed image, further cuts out the collationimage as a collation area from the area of the ink portion, andtransmits the collation image to the server computer 50 via thecommunication network. The processing of the processor of the collationimage photographing machine 22 will be further described later.

The server computer 50 includes a collation unit 50 a and the registeredimage DB 50 b.

The registered image DB 50 b is composed of a storage device such as ahard disk or a solid state drive (SSD), and stores an identifier ID foruniquely specifying the object 10 and the registered image inassociation with each other.

The collation unit 50 a is composed of a processor, and stores theregistered image received from the registered image photographingmachine 20 in the registered image DB 50 b in association with the ID ofthe object 10. Further, the collation unit 50 a performs image collationbetween the collation image received from the collation imagephotographing machine 22 and the registered image stored in theregistered image DB 50 b, and outputs the collation result to thecollation image photographing machine 22. Specifically, the collationunit 50 a reads the registered image from the registered image DB 50 b,performs collation calculation with the collation image, and calculatesthe degree of similarity between the two images. For the calculation ofthe degree of similarity, a known algorithm such as feature amountmatching by feature amount detection, template matching using imageshading comparison, and the like can be used. The calculated degree ofsimilarity is compared with a threshold, and in a case where the degreeof similarity exceeds the threshold, it is determined that the bothmatch, and in a case where the degree of similarity does not exceed thethreshold, it is determined that the both do not match. The collationunit 50 a transmits the collation result to the collation imagephotographing machine 22 via the communication network.

In the image collation, there is an error rate due to fluctuations inthe input of the registered image photographing machine 20 or thecollation image photographing machine 22 to an image sensor,quantization error, and the like. The error rate consists of two, afalse rejection rate, which is the probability of determining false eventhough it is true, and a false acceptance rate, which is the probabilityof determining true even though it is false. The two are in a trade-offrelationship, and in a case where one decreases, the other increases.Therefore, a threshold is set so that the loss in the application targetof the collation determination is the smallest.

Note that a plurality of the registered images may be acquired bychanging the irradiation direction of light and registered in theregistered image DB 50 b of the server computer 50, and the imagecollation between the plurality of registered images and the collationimage may be performed.

FIG. 2 shows a block diagram of a main configuration of a collationimage photographing machine 22 such as a smartphone. The collation imagephotographing machine 22 includes a processor 22 c, a ROM 22 d, a RAM 22e, an input unit 22 f, an output unit 22 g, and a communication I/F 22 hin addition to the light source unit 22 a and the camera unit 22 bdescribed above.

The processor 22 c reads out an application program stored in the ROM 22d, executes series of processing using the RAM 22 e as a working memory,extracts the ink portion from the photographed image photographed by thecamera unit 22 b, and further cuts out the collation image. Theprocessor 22 c transmits the cut-out collation image to the servercomputer 50 via the communication I/F 22 h. Further, the processor 22 creceives the collation result from the server computer 50 via thecommunication I/F 22 h.

The input unit 22 f is composed of a keyboard, a touch switch, or thelike, and the user operates the input unit 22 f to start the applicationprogram.

The output unit 22 g is composed of a liquid crystal display, an organicEL display, or the like and functions as a display unit, and displays apreview image in a case where the object 10 is photographed. Further,the output unit 22 g displays a guide in a case of photographing theobject 10 by a control command from the processor 22 c. The guide willbe further described later. Further, the output unit 22 g displays thecollation result received from the server computer 50 by the controlcommand from the processor 22 c. The collation result is either “match”or “mismatch”, but other messages related to the collation may bedisplayed.

FIGS. 3A and 3B show plan diagrams of the object 10 in the presentexemplary embodiment. FIG. 3A shows a case where a hologram label sealis used as the object 10. The hologram label seal includes hologramportions 12 a and 12 b, a QR code 13, and an ink portion 14.

The hologram portion 12 a is formed substantially on a left half of thelabel seal and forms a hologram pattern. The hologram portion 12 b isformed substantially on a right half of the label seal and is subjectedto satin treatment, so that color development of a rainbow color ischanged depending on an elevation angle. Here, the “elevation angle”means an angle formed by an LED light source 22 a, the object 10, andthe camera unit 22 b in FIG. 1 . For example, it is desirable that theelevation angle matches an angle φ in FIG. 1 .

The QR code 13 is formed on the satin-treated hologram portion 12 b. Forthe QR code 13, various information about the label seal is printed as aQR code. Further, in the present exemplary embodiment, the QR code 13 isprinted with the relative positional relationship fixed in advance withrespect to the ink portion 14. Therefore, the QR code 13 can alsofunction as a positioning mark for extracting the ink portion 14.

The ink portion 14 is gravure-printed with gravure ink containing silverparticles on the satin-treated hologram portion 12 b to form a polygonalshape. In the figure, the ink portion 14 is printed in a square shape ata lower part of the QR code 13 at a fixed interval. The ink portion 14is a random pattern area, which is a collation area to be photographedand extracted by the collation image photographing machine 22. The shapeof the ink portion 14 may be an ellipse (including a circle) as well asa polygon.

FIG. 3B shows a case where a paper label seal is used as another object10. The paper label seal includes a paper portion 11, the QR code 13,and the ink portion 14.

The QR code 13 is formed on the paper portion 11. For the QR code 13,various information about the label seal is printed as a QR code. The QRcode 13 is printed with the relative positional relationship withrespect to the ink portion 14 fixed in advance. Therefore, the QR code13 can also function as a positioning mark for extracting the inkportion 14.

The ink portion 14 is toner-printed on the paper portion 11 with tonerink containing silver particles to form a polygonal shape. In thefigure, the ink portion 14 is printed in a square shape at a lower partof the QR code 13 at a fixed interval. The ink portion 14 is a randompattern area, which is a collation area to be photographed and extractedby the collation image photographing machine 22.

FIG. 4 schematically shows position specifying processing of the inkportion 14 using the relative positional relationship between the QRcode 13 and the ink portion 14. In order to extract the ink portion 14,the QR code 13 is first detected. In a case where the QR code 13 isdetected, the position of the ink portion 14 is specified by using aknown relative positional relationship between the QR code 13 and theink portion 14 (the relative positional relationship is indicated by avector in the figure). Then, the square ink portion 14 is extracted fromthe photographed image at the specified position.

However, as described above, since a random pattern of the ink portion14 changes depending on a light irradiation direction, the irradiationangle φ of the irradiation light from the light source unit 22 a needsto be set to be substantially the same as the angle φ which is thecondition in a case where the registered image is acquired. Theexpression “substantially the same” means a permissible range thatincludes deviations from the desired angle required to ensure thecollation accuracy. In a case where the user uses a mobile terminal suchas a smartphone as the collation image photographing machine 22, it isrelatively difficult to set the elevation angle φ to a desired angle dueto a manual operation. Although it is possible to conceive of a methodof automatically repeating the photographing and adopting an image witha good condition, unless a photographer knows whether or not theelevation angle is appropriate, it is not possible to acquire an imagewith a good condition in a short time.

FIGS. 5A and 5B schematically show a positional relationship between thecollation image photographing machine 22 and the object 10. FIG. 5Ashows a case where the elevation angle φ formed by the light source unit22 a, the object 10, and the camera unit 22 b is a desired angle, thatis, substantially the same as the elevation angle in a case where theregistered image is photographed.

On the other hand, FIG. 5B shows a case where the elevation angle formedby the light source unit 22 a, the object 10, and the camera unit 22 bdeviates from the angle φ and is substantially different from thedesired angle. Even in a case where the user tries to set the elevationangle to the desired angle by the manual operation, it is unclear inwhich direction the elevation angle should be adjusted, unless there issome kind of guide, which not only takes time but also reduces thecollation accuracy.

Therefore, in the present exemplary embodiment, in a case where thepositional relationship among the light source unit 22 a, the object 10,and the camera unit 22 b is inappropriate and the elevation angle isdifferent from the desired angle as shown in FIG. 5B, a guide isdisplayed to notify the user of the fact and in which direction theelevation angle should be adjusted.

FIG. 6 schematically shows a preview image 30 displayed on the outputunit 22 g of the collation image photographing machine 22 such as asmartphone.

In FIG. 6 , an x mark located at a center of the preview image 30indicates a center of the camera unit 22 b. Further, a mark 32 locatedon an upper left from the center indicates an actual position of thelight source unit 22 a. The light source unit 22 a and the camera unit22 b are located on a straight line shown by a broken line in thefigure, and in FIG. 6 , it is shown that the light from the light sourceunit 22 a emits the ink portion 14 from the upper left.

At this time, the preview image 30 has a bright spot of the light sourceunit 22 a, that is, a specular reflection portion of the light from thelight source unit 22 a, and in a case where the object 10 is placed on ahorizontal plane and the collation image photographing machine 22 suchas a smartphone is held at a position horizontal to face the object 10,the bright spot of the light source unit 22 a is located at anintermediate point between the position of the light source unit 22 aand a center position of the camera unit 22 b. Since a distance betweenthe position of the light source unit 22 a and the center position ofthe camera unit 22 b is known for each model of the collation imagephotographing machine 22, the intermediate point between the position ofthe light source unit 22 a and the center position of the camera unit 22b is uniquely determined. In FIG. 6 , the position of the intermediatepoint is shown as a guide 34. Therefore, assuming that the elevationangle φ becomes substantially the same as the desired angle by adjustinga posture so that a surface (a surface on which the light source unit 22a and the camera unit 22 b are mounted) of the collation imagephotographing machine 22 is horizontal to a surface of the object 10,the user can easily adjust the elevation angle to the desired angle byadjusting the elevation angle φ so that the position of the bright spotof the light source unit 22 a appearing on the preview image 30 matchesthe guide 34 displayed on the preview image 30.

Further, since the random pattern of the ink portion 14 can bephotographed satisfactorily in a case where the elevation angle φ issubstantially the same as the desired angle, a guide 14 g indicating theposition of the ink portion 14 is displayed together with the guide 34.That is, since the QR code 13 is detected from the preview image 30 andan existing position of the ink portion 14 is specified by using theknown relative positional relationship from the QR code 13, the guide 14g indicating the area of the ink portion 14 is displayed at thespecified existing position. Hereinafter, the guide displayed on thepreview image 30 will be described in more detail.

FIG. 7 schematically shows a photographing posture of the collationimage photographing machine 22 such as a smartphone with respect to theobject 10. As shown in FIG. 7 , the light source unit 22 a and thecamera unit 22 b of the collation image photographing machine 22 areprovided so as to be arranged in a vertical direction on a back surfaceside. The light source unit 22 a is provided on a lower side in thevertical direction, and the camera unit 22 b is provided on an upperside.

Note that although not shown in FIG. 7 , the output unit 22 g as adisplay unit is provided on a front surface side of the collation imagephotographing machine 22, and the photographed image of the camera unit22 b is displayed as a preview image.

As shown in FIG. 7 , the user grips the collation image photographingmachine 22 so that an arrangement direction (vertical direction) of thelight source unit 22 a and the camera unit 22 b matches an arrangementdirection of the QR code 13 and the ink portion 14 of the object 10, andmanually operates so that the collation image photographing machine 22and the object 10 are substantially parallel to each other and thecamera unit 22 b faces the ink portion 14. The arrow in FIG. 7 shows howthe camera unit 22 b is manually moved so as to face the ink portion 14while maintaining the collation image photographing machine 22 parallelto the object 10.

FIG. 8 shows the preview image 30 displayed on the output unit 22 g ofthe collation image photographing machine 22 in the photographingposture of FIG. 7 . The object 10, the QR code 13, and the ink portion14 are displayed on the preview image 30. As an example, the object 10shows a case where the printing substrate is the paper 11 as shown inFIG. 3B. Further, in a case where the surface provided with the lightsource unit 22 a and the camera unit 22 b is parallel to the surface ofthe object 10, the guide 34 indicating a position of a bright spot isdisplayed in a double circle format at a position of an intermediatepoint between the light source unit 22 a and the camera unit 22 b.Further, along with this, a guide 13 g indicating a position of the QRcode 13 and a guide 14 g indicating a position of the ink portion 14 aredisplayed.

The user adjusts the elevation angle of the collation imagephotographing machine 22 so that the position of the bright spot of thelight source unit 22 a reflected in the preview image 30 is aligned withthe position of the guide 34. In a case where the position of the brightspot matches the position of the guide 34, it can be recognized that theelevation angle of the collation image photographing machine 22substantially matches the desired angle.

By displaying the guide 34 indicating the position of the bright spot inthis way, the user may visually confirm whether or not the elevationangle is close to the desired angle, so that a good random pattern imageof the ink portion 14 may be acquired. Note that although the shape ofthe ink portion 14 is known (square in the present exemplaryembodiment), and it can be assumed that the deviation of the elevationangle is confirmed by using distortion from the known shape, since thedistortion of the shape due to the change in elevation angle is lessthan 1% per degree and cannot be visually determined, visual inspectionby the guide 34 is effective.

In FIG. 8 , the guide 13 g and the guide 14 g are displayed in additionto the guide 34, but at least one of the guide 13 g or the guide 14 gmay be displayed. Further, the guide 13 g may be displayed in a casewhere the QR code 13 is detected, and the guide 14 g may be displayed ina case where the elevation angle φ is substantially the same as thedesired angle.

Further, FIG. 8 shows a case where the printing substrate is the paper11 as the object 10 as shown in FIG. 3B, in a case where the printingsubstrate is the hologram portion 12 b as the object 10 as shown in FIG.3A, the characteristic of the reflected light changes depending on thedirection of light, and the color development pattern changes, and thusa guide of an elevation angle can be displayed by using the change inthe color development pattern.

FIG. 9 shows another preview image 30 displayed on the output unit 22 gof the collation image photographing machine 22 in the photographingposture of FIG. 7 . The object 10, the QR code 13, and the ink portion14 are displayed on the preview image 30. As shown in FIG. 3A, theobject 10 shows a case where the printing substrate is the hologramportion 12 b.

The QR code 13 and the ink portion 14 are printed on the satin-treatedhologram portion 12 b, and since the hologram portion 12 b changes thecolor development pattern in accordance with the light irradiationdirection, that is, the elevation angle φ, in a case where focusing onthe periphery of the ink portion 14, the color development patternaround the ink portion 14 changes in accordance with the elevation angleφ. Therefore, the color development pattern (or color or pattern) can beused as a guide of the elevation angle.

In addition to the color development pattern (or color or pattern), amessage about the color development pattern may be displayed at a fixedposition on the preview image 30. For example, in a case where the colordevelopment pattern around the ink portion 14 becomes the rainbow colorin a case where the elevation angle φ substantially matches the desiredangle, a message such as “please match the rainbow color with thesample” is displayed. By displaying the rainbow color around the inkportion 14 and matching the actual color development pattern by usingthe rainbow color as a guide 36, the user can visually confirm thedeviation of the elevation angle.

Note that in FIG. 9 , in a case where the printing substrate of theobject 10 is the hologram portion 12 b, although the color developmentpattern (color or pattern) is displayed on the preview image 30 as theguide 36 of the elevation angle, in this case as well, the guide 34indicating the position of the bright spot may be displayed as the guideof the elevation angle as in the case where the printing substrate isthe paper 11, and at least one of the guide 34 or the guide 36 may bedisplayed.

Further, the guides 34 and 36 of the elevation angle may be displayed,the current elevation angle and the desired elevation angle may becompared, and a message may be displayed as to which direction the usershould adjust the elevation angle.

FIGS. 10A to 10C show examples of displaying a guide of an elevationangle and a message. FIG. 10A shows the preview image 30 in a case wherethe position of the bright spot of the light source unit 22 a matchesthe guide 34. The processor 22 c detects the position of the bright spotreflected in the light source unit 22 a in the preview image 30, andcompares the detected position with the position of the guide 34. Then,it is determined that the elevation angle is substantially the same asthe desired angle in a case where the both positions match within apermissible range, and a mark 38 indicating that the photographingposture is appropriate is displayed on the preview image 30. In thefigure, the mark is a “check mark”, but is not limited thereto.

FIGS. 10B and 10C show the preview image 30 in a case where the positionof the bright spot of the light source unit 22 a does not match theguide 34. The processor 22 c detects the position of the bright spotreflected in the light source unit 22 a in the preview image 30, andcompares the detected position with the position of the guide 34. Then,it is determined that the elevation angle is not substantially the sameas the desired angle in a case where the both positions do not matchwithin the permissible range, and the mark 38 indicating that thephotographing posture is appropriate is not displayed, and a message 40indicating a direction in which the elevation angle should be adjustedin accordance with the direction of deviation between the two positionsis displayed on the preview image 30. In FIG. 10B, since the position ofthe bright spot is located at a lower right side of the guide 34, inorder to correct the position of the bright spot, the message 40, whichis a message “Head UP” instructing a head portion (head) of thecollation image photographing machine 22 to face upward is displayed.Further, in FIG. 10C, since the position of the bright spot is locatedon an upper left side of the guide 34, in order to correct the positionof the bright spot, the message 40, which is a message “Bottom UP”instructing the bottom portion (bottom) of the collation imagephotographing machine 22 to face upward is displayed.

In FIGS. 10A to 10C, although the position of the bright spot and theposition of the guide 34 are compared, in a case where the colordevelopment pattern is displayed as the guide 36 as shown in FIG. 9 ,the color development pattern in the vicinity of the ink portion 14 ofthe preview image 30 may be detected and compared with the colordevelopment pattern of the guide 36. Specifically, the image around theink portion 14 is acquired, the center of gravity position of a specificcolor (for example, red, yellow, and light blue) is calculated bybinarizing the color-decomposed image, and it may be determined whetheror not the calculated center of gravity position is within the presetthreshold.

FIGS. 11A to 11C show examples of change in the color developmentpattern around the ink portion 14 in the preview image 30. FIG. 11A is acolor development pattern in a case where the elevation angle matchesthe desired elevation angle. On the other hand, FIGS. 11B and 11C showcolor development patterns in a case where the elevation angle does notmatch the desired elevation angles. The processor 22 c may calculate thecenter of gravity position of a specific color (for example, red,yellow, and light blue) by binarizing the color-decomposed image, and ina case where the calculated center of gravity position is not within thepreset threshold, display a message such as “The rainbow color does notmatch. Please raise the angle toward you.” to instruct the user in thedirection of adjusting the elevation angle.

FIG. 12 shows a processing flowchart of the processor 22 c. This isprocessing realized by reading and executing a program stored in the ROM22 d or the like.

First, the processor 22 c acquires a photographed image obtained by thecamera unit 22 b (S11).

Next, the guide of the elevation angle is displayed by superimposing theguide of the elevation angle on the acquired photographed image on thepreview image 30 (S12). The guide of the elevation angle is at least oneof the guide 34 indicating the position of the bright spot in a casewhere the elevation angle φ substantially matches the desired angle, orthe guide 36 of a light emitting pattern. Along with the guides 34 and36, messages such as “Please match the bright spot of the light sourceunit 22 a with the guide.”, “Please match the rainbow color with thesample.”, and the like may be displayed. These messages can alsofunction as the guide of the elevation angle.

Next, it is determined whether or not the actual elevation angle of thecollation image photographing machine 22 is appropriate, that is,whether or not the elevation angle φ is substantially the same as thedesired angle (S13). In a case where the guide 36 is displayed, theprocessor 22 c detects the actual bright spot of the light source unit22 a in the preview image 30, compares the position of the detectedbright spot with the position of the guide 34, determines that theelevation angle is appropriate in a case where the both match within thepermissible range, and determines that the elevation angle is notappropriate in a case where the both do not match within the permissiblerange. Further, in a case where the guide 36 is displayed, the processor22 c determines that the elevation angle is appropriate in a case wherethe center of gravity position of the specific color is within thepreset threshold range, and determines that the elevation angle is notappropriate otherwise. In a case where the elevation angle is notappropriate (NO in S13), the processing after S11 is repeatedlyexecuted.

On the other hand, in a case where it is determined that the elevationangle is appropriate (YES in S13), the ink portion 14 is detected byusing a known relative positional relationship with reference to the QRcode 13. That is, the shape of the ink portion 14 (square in the presentexemplary embodiment) is extracted (S14).

FIG. 13 shows a detailed flowchart of the shape extraction.

The purpose of the processing flowchart is to acquire coordinates of thefour vertices of the ink portion 14 of the square from the photographedimage, and the processing is substantially classified into threeprocessing of binarized image generation processing (S1), rectangularedge extraction processing (S2), and vertex coordinate estimationprocessing (S3).

Binarized Image Generation Processing

First, the binarized image generation processing (S1) will be described.

In the processing, first, smoothing processing and shading differenceenhancement processing are simultaneously executed on the original image(S101). The shape of the ink portion 14 is blurred by simply performingthe smoothing processing on the original image. Further, although theunevenness of the ink portion 14 is emphasized by simply performing theshading difference enhancement processing on the original image, theunevenness of the hologram portion 12 b is also emphasized at the sametime, so that the ink portion 14 cannot be extracted.

Therefore, the smoothing processing and the shading differenceenhancement processing are simultaneously executed on the original imageto remove the unevenness of the hologram portion 12 b and the inkportion 14, and the ink portion 14 is identified from the hologramportion 12 b. Specifically, a mean-shift filter can be used forsimultaneous execution of the smoothing processing and the shadingdifference enhancement processing. The mean-shift filter is realized bythe processor 22 c. The mean-shift filter is a filter that fills similarcolors in a designated pixel space with the same color. As a result, thecolor of the silver ink portion 14 approaches the same color, and aboundary between a rainbow-colored background of the hologram portion 12b and the silver ink portion 14 has a different color area, so that ashading difference of the boundary between the hologram portion 12 b andthe ink portion 14 is emphasized while the shape of the ink portion 14is maintained.

Note that although there is a filtering method of performing smoothingprocessing while retaining the edges, such as a bilateral filter, theinventors have confirmed that the noise in the hologram portion 12 b andthe ink portion 14 cannot be removed by the method. By using themean-shift filter, smoothing is performed for each color by using thecolor difference between the hologram portion 12 b and the ink portion14 while retaining the edges, and noise may be removed without losingthe edges.

The mean-shift filter first searches for the center of gravity of thecolor distribution of the original image. That is, centroid coordinates(xc, yc) and the colors (rc, gc, bc) of a color space area having aradius sr centered on the colors (r, g, b) of certain pixels (x, y) arecalculated and the center of gravity is searched under the followingcondition. Here, sp is a radius of a search area.

Condition: |x−xc|≤sp, |y−yc|≤sp, ∥(r, g, b)−(rc, gc, bc)∥≤sr

Then, in a case where the above condition is satisfied,

by setting (x, y, r, g, b)=(xg, yg, rc, gc, bc),

the center of gravity is searched again. The above center of gravitysearch processing is repeatedly executed.

Then, a color space distance ε and the number of repetitions n are setin advance, it is determined whether or not the following condition issatisfied, and in a case where the condition is satisfied, theprocessing ends.

Condition: the number of repetitions n is satisfied,

or

|x−xc|+|y−yc|+(r−rc)²+(g−gc)²+(b−bc)²<ε.

After the center of gravity search processing ends, smoothing isperformed with the value of the center of gravity in the color space.That is, after the polar search ends, each pixel in the space is set asa center of gravity value of the color space. The edges are thenclarified by using the Gaussian pyramid and the threshold sr.

Since the mean-shift filter performs the smoothing processing by using adistance difference in the color space, this is an effective smoothingprocessing in a case where there is a difference in the color spacedistance between the foreground and the background. Therefore, this isan effective processing for the original image in which an achromaticink portion 14 exists in the foreground and a chromatic hologram portion12 b exists in the background.

In the mean-shift filter, the performance of the smoothing processingand the shading difference enhancement processing can be controlled byusing a color space radius sr and a pixel space radius sp as majorparameters. Therefore, by adjusting the parameters, a ratio of thesmoothing processing and the shading difference enhancement processingcan be adjusted. Specifically, (1) since a search range of pixelssmoothed (filled) by the pixel space radius sp is designated,

in a case where sp is large→search range can be adjusted to be wide, and

in a case where sp is small→search range can be adjusted to be narrow.

Note that in a case where the sp is set too large, it takes a long timeto processing, for example, so it is desirable to take the fact intoconsideration.

(2) Since the range of similar colors to be filled in the same color isdetermined by the color space radius sr,

in a case where sr is large→it can be adjusted so that even slightlydifferent colors are recognized as the same color, and

in a case where sr is small→it can be adjusted so that similar colorsare recognized as the same color.

In the present exemplary embodiment, the parameters sp and sr of themean-shift filter are set to

(sp, sr)=(10,30), or the like.

After the smoothing processing and the shading difference enhancementprocessing are simultaneously executed on the original image (S101), anadditional shading difference enhancement processing is further executedfor a portion where the shading difference cannot be obtained by theprocessing of S101 (S102).

In a case where there is no sufficient difference in the color spacedistance between the background color and the ink portion 14, theforeground and the background are assimilated, and there may be portionswhere the shading difference enhancement is insufficient only by theprocessing of S101. Therefore, by further executing the shadingdifference enhancement processing, the shape of the ink portion 14 isextracted more stably.

Specifically, the image processed in S101 is RGB-decomposed, and theshading difference enhancement processing is executed in each RGB colorspace. This means flattening of an in-image brightness histogram. Then,in order to extract an edge gradient, a Sobel filter for each of thevertical and horizontal directions is applied to each RGB color image.Note that since a gradient value calculated by the Sobel filter is noteight bits (256 gradations), it may be normalized to eight bits. Thenormalization method is processing of taking an absolute value of agradient image and replacing all the pixel values of 255 or more with255. As a result, the edge gradient may be acquired independently ofdisturbance noise.

After executing additional shading difference enhancement processing(S102), noise removal processing using an HSV color space is executed(S103). Here, the HSV color space is a color space composed of threecomponents of Hue, Saturation/Chroma, and Value/Brightness.

In a case where a rough shape of the ink portion 14 is extracted inS102, noise is generated at a boundary between the white color and thelight blue color of the hologram portion 12. In particular, since thegradients of the white color and the light blue color in an R spaceimage are large, noise such as an edge is generated in a case where theSobel filter is applied. Therefore, the noise is removed by using theHSV color space. Specifically, (1) the processed image 13 in S101 is HSVdecomposed (2) S image is binarized (3) vertical and horizontal Sobelfilter is applied to S binary image (4) black-and-white inverted binaryimage of vertical and horizontal Sobel image and H image areOR-synthesized.

After executing the processing of S103, a binarized image is created(S104). That is, a total of 6 vertical and horizontal gradient images ofthe R, G, and B images are respectively binarized. A binarizationthreshold may be set differently for each of R, G, and B. Then, a totalof six binarized images of vertical and horizontal components and RGBcolor components are OR-synthesized.

Rectangular Edge Extraction Processing

Next, the rectangular edge extraction processing will be described.

After creating the binarized image in S104, the sides of the polygonconstituting the square ink portion 14 are acquired from the binarizedimage (S105). Specifically, this is edge extraction processing using astochastic Hough transform. Note that the stochastic Hough transform isan optimization of the Hough transform, and instead of calculating usingall the pixels, points sufficient for straight line detection arerandomly selected from the image and calculated. A (non-stochastic)Hough transform can also be used in S104. However, parameter tuning isdifficult, and there is a drawback that the sensitivity is too good forthe rectangular edges of the binarized image.

After acquiring the sides of the polygon (S105), the processing ofremoving extra sides is executed (S106). That is, straight lines thatare not rectangular edges (sides) are removed from the straight linesextracted by the stochastic Hough transform. Specifically, a method ofremoving a straight line having an inclination of a fixed value or more,or a long straight line with respect to the size of the collation area,a straight line of which an angle in which vertical and horizontal linesintersect is within a fixed angle, or removing a straight line incontact with an image frame is used. In addition to this, extra sidesmay be removed by extracting edges with a rectangular hue by using acolor space.

Vertex Coordinate Estimation Processing

After the rectangular edge extraction processing (S2) is completed, thevertex coordinate estimation processing (S3) of the square ink portion14 is executed.

In the processing, the centroid coordinates of the intersections of thesides are calculated from the image obtained by removing the extra sidesin S106 (S107). That is, instead of the intersections consisting of eachside, the centroid coordinates of an intersection group within a certainvicinity are calculated. Although the intersections of vertical andhorizontal straight lines are calculated for the processed image 23 byaddressing a one-dimensional simultaneous equation, since an edge widthof the binarized image after OR-synthesis is two to three pixels, aplurality of straight lines are extracted for the same edge by thestochastic Hough transform. Therefore, there are a plurality ofintersections in the vicinity of certain coordinates. Since theseintersections are likely to indicate the same vertex, the centroidcoordinates of the intersection group are acquired, and the centroidcoordinates are redefined as the vertices of the shape of the inkportion 14.

In centroid coordinate calculation processing of the intersection group,first, a plurality of intersections in a certain vicinity are dilatedand combined into one. The dilation processing is processing in which ina case where there is a white pixel in peripheral pixels of a certainpixel, the pixel is converted into the white pixel thereby sequentiallyexpanding the white pixel. Next, labeling is performed on eachintersection set that has been dilated. Then, the centroid coordinatesof each labeled intersection set are calculated. In a case where thecentroid coordinates are calculated as described above, the calculatedcentroid coordinates are set as the vertex candidates.

Since there are four vertices in the square ink portion 14, four vertexcandidates can be set normally. In a case of setting the vertexcandidates, known shape characteristics of the ink portion 14, that is,the lengths of the sides and diagonal lines can be used as thecondition. In a case where there are a plurality of pairs of verticesthat satisfy the condition, a plausible pair of vertices is selected.For example, in the square ink portion 14, the condition that thelengths of the four sides are equal to each other is used, and the pairhaving the smallest dispersion of the side lengths is set as the pair ofvertices.

Then, it is determined whether or not all the vertices of the inkportion 14 have been acquired (S108). In the square ink portion 14, itis determined that all the vertices have been acquired in a case wherethe four vertices are acquired. In a case where all the vertices havenot been acquired, it means that all the sides of the ink portion 14have not been extracted, so the complementary processing of thedisappeared side is executed next (S109).

In the complementary processing of the side, it is determined whether ornot three sides constituting the square have been extracted from thesquare ink portion 14. Normally, in a case where the ink portion 14 isprinted on the hologram portion 12 b as a printing substrate, in a casewhere the red of the background of the hologram portion 12 is coveredwith the red of the foreground, extraction of the side may fail. Inshort, this is a case where a color space distance difference betweenthe background and the foreground is small.

Therefore, it is first determined whether or not the three sides havebeen extracted. The selection of the three sides can be estimated fromthe known shape characteristics of the ink portion 14, that is, thelength or the position of the edges.

In a case where three sides are extracted, a length x of the side amongthe three sides having no opposite side is calculated from the centroidcoordinates already calculated. Then, a new parallel side is drawn at aportion separated by the length x of the side. Specifically, it isassumed that the four sides constituting the square are a, b, c, and d,a and c are opposite sides, and b and d are opposite sides, and in acase where only three sides of a, b, and c are extracted, a sideparallel to b is drawn at a position separated by x from b to be d. Asparallel sides separated by x from b, it is possible to estimate a totalof two, one on each of the both sides of b, and since one of the sidesdoes not exist in the image, the side d can be uniquely drawn. Thiscomplements the disappeared sides.

After the complementary processing of the disappeared sides, thecentroid coordinates of the intersection may be calculated again toacquire the coordinates of all the vertices.

Note that after the complementary processing of the disappeared sides,the threshold may be lowered for the binarized image obtained in S104,and the stochastic Hough transform may be executed again to reacquirethe side, and the side obtained in this way and the side obtained bycomplementing may be integrated, and the processing may be transferredto the vertex coordinate estimation processing (S3) again.

Referring back to FIG. 12 , in a case where the shape is extracted byacquiring the coordinates of the four vertices of the ink portion 14 asdescribed above (YES in S14), the processor 22 c acquires a randompattern image by cutting out the collation image with reference to thecoordinates of these four vertices (S15). Then, an image qualityevaluation of the obtained random pattern image, that is, whether or notthe image quality of the random pattern image is good is determined(S16).

Specifically, whether or not the image quality is good can be determinedby evaluating the following index values and determining whether or notthese index values exceed the threshold.

(1) Whether the position, size, and angle of the square are appropriate

(2) Degree of blur (standard deviation of Laplacian filter value)

(3) Degree of shaking (maximum and minimum differences of standarddeviation of Sobel filter values in four directions)

(4) Brightness (average brightness)

(5) Randomness (a value obtained by cutting out the portion of thecentral ¼ size of the image, obtaining the correlation value betweeneach coordinate of the image and the image of the same size as thestarting point, subtracting the average value from the maximum value ofthe correlation value group, and dividing by the standard deviation)

(6) Degree of deviation of the light source (aspect ratio of thebrightness inclination of the image=the inclination in a case where theaverage brightness of the same row is linearly approximated in thecolumn direction/the inclination in a case where the average brightnessof the same column is linearly approximated in the row direction)

It is possible to determine whether or not the image quality is good byoptionally combining any one or a plurality of these index values. Forexample, (1) and (6) are used, (1), (5), and (6) are used, or the like.

The processing of S11 to S16 is automatically repeated until apredetermined upper limit number of N images are acquired, until atimeout, or until the user interrupts the photographing operation (S17).Then, the collation processing is executed by using the random patternimages obtained for the upper limit number of N images or a plurality ofrandom pattern images obtained until the timeout or the interruption ofphotographing (S18).

In the collation processing, a collation request is transmitted to theserver computer 50 with the acquired upper limit number of N images or aplurality of random pattern images attached. The collation unit 50 a ofthe server computer 50 collates the received random pattern image withthe registered image, and returns the collation result to the collationimage photographing machine 22. The processor 22 c receives thecollation result from the server computer 50 and displays the collationresult on the output unit 22 g.

Modified Example

In the exemplary embodiment, as shown in FIG. 7 , a configuration inwhich the ink portion 14 is irradiated with light of the light sourceunit 22 a from one direction on a plane and photographed by the cameraunit 22 b has been described as an example. However, since the minuteunevenness random pattern of the ink portion 14 changes depending on thelight irradiation direction, by registering a first registered image ina case where light is emitted from a first direction on a plane and asecond registered image in a case where light is emitted from a seconddirection different from the first direction on a plane in theregistered image DB 50 b as the registered images, and collating thecollation image obtained by emitting light from the first direction on aplane with the first registered image in the first collation, andcollating the collation image obtained by emitting light from the seconddirection on a plane with the second registered image in the secondcollation, a so-called double check can be performed to further improvethe collation accuracy.

FIG. 14 schematically shows a positional relationship between thecollation image photographing machine 22 such as a smartphone and theobject 10. (a) of FIG. 14 is the same as in the case of FIG. 7 , and thephotographing posture of the collation image photographing machine 22 isset so that the light from the light source unit 22 a is emitted from alower side of the ink portion 14 on a plane.

In (b) of FIG. 14 , the operating posture of the collation imagephotographing machine 22 is set so that the object 10 is not moved froma case of (a) of FIG. 14 and the posture of the collation imagephotographing machine 22 is rotated 90 degrees clockwise on a plane andthe light from the light source unit 22 a is emitted from the left sideof the ink portion 14 on a plane.

A double check is performed by cutting out the collation image so thatthe first direction of the first collation is the direction shown in (a)of FIG. 14 and collating the collation image with the first registeredimage, and cutting out the collation image so that the second directionof the second collation is the direction shown in (b) of FIG. 14 andcollating the collation image with the second registered image.

At this time, since the guide of the elevation angle has a function ofnotifying the user of the light irradiation direction, by displaying theguide 34 at the position of the bright spot in a case of emitting fromthe lower side of the ink portion 14 at the time of the first collation,and displaying the guide 34 at the position of the bright spot in a caseof emitting from the left side of the ink portion 14 at the time of thesecond collation, it is possible to accurately notify the user of thechange in the light irradiation direction from the light source unit 22a of the collation image photographing machine 22 in the first andsecond collation.

FIG. 15 shows a processing flowchart of the processor 22 c of themodified example. This is processing realized by reading and executing aprogram stored in the ROM 22 d or the like.

First, a top page is displayed on the output unit 22 g (S201). Variousmenus are displayed on the top page. The menu includes “startcollation”, and the user selects and operates the start of collation tostart the subsequent processing.

First, the processor 22 c photographs a random pattern image in thefirst direction on a plane (S202). That is, for example, the firstdirection is set as the lower side of the ink portion 14, and S11 to S17of the processing flowchart shown in FIG. 12 are executed to acquire therandom pattern image in the first direction. Then, a collation requestis made by attaching the random pattern image with respect to the servercomputer 50 (S203), and the collation result is received.

As a result of collation (S204), in a case where the random patternimage does not match the first registered image, an error display isperformed on the output unit 22 g (S209), and the processing after S201is repeated. Further, in a case where the image quality of the randompattern image does not reach the image quality required for collation,processing of S202 is executed after displaying to that effect, and therandom pattern image in the first direction is re-photographed.

As a result of collation, in a case where the random pattern imagematches the registered image (OK in S204), the processor 22 c performs adisplay to that effect and then displays a selection button as towhether or not to perform additional collation. In a case where the userdoes not execute the additional collation, the processing ends (S205).

In a case where the user executes the additional collation (YES inS205), the processor 22 c photographs the random pattern image in thesecond direction on a plane (S206). That is, for example, the seconddirection is set perpendicular to the first direction and set as theleft side of the ink portion 14, and S11 to S17 of the processingflowchart shown in FIG. 12 are executed to acquire the random patternimage in the second direction. Then, a collation request is made byattaching the random pattern image with respect to the server computer50 (S207), and the collation result is received.

As a result of collation (S208), in a case where the random patternimage does not match the second registered image, an error display isperformed on the output unit 22 g (S209), and the processing after S201is repeated. Further, in a case where the image quality of the randompattern image does not reach the image quality required for collation,processing of S206 is executed after displaying to that effect, and therandom pattern image in the first direction is re-photographed. In acase where both the first collation and the second collation match theregistered image (OK in S208), a message to that effect is displayed asthe collation is successful, and the processing ends.

FIG. 16 shows first collation processing. (a) of FIG. 16 shows thepreview image 30, and the object 10, the guide 13 g, the guide 14 g, andthe guide 34 are displayed. The guide 34 indicating the elevation angleis displayed on the lower side of the ink portion 14. (b) of FIG. 16shows the preview image 30 in a case where the user adjusts thephotographing posture of the collation image photographing machine 22 sothat the bright spot of the light source unit 22 a matches the guide 34,and the mark 38 is displayed in response to the fact that both positionsmatch (refer to FIG. 10A). At this time, the photographed image isautomatically acquired, the random pattern image is cut out, and thephotographed image is transmitted to the server computer 50 as thecollation image for collation. During the collation, a ring-shaped mark42 is displayed. By visually recognizing the mark 42, the user canconfirm that the image is being collated by the server computer 50. (c)of FIG. 16 is the preview image 30 in a case where the first collationis successful, and the message 44 “The collation is successful.” isdisplayed. Further, along with the message 44, messages 46 and 48 of“additional collation” and “end” are displayed. In a case where the userselects the message 46 of “additional collation”, the screen transitionsto the screen of FIG. 17 .

FIG. 17 shows second collation processing. (a) of FIG. 17 shows thepreview image 30, and the object 10, the guide 13 g, the guide 14 g, andthe guide 34 are displayed. The guide 34 indicating the elevation angleis displayed on the left side of the ink portion 14. (b) of FIG. 17shows the preview image 30 in a case where the user rotates and adjuststhe photographing posture of the collation image photographing machine22 by 90 degrees so that the bright spot of the light source unit 22 amatches the guide 34, and the mark 42 is displayed in response to thefact that both positions match. At this time, the photographed image isautomatically acquired, the random pattern image is cut out, and thephotographed image is transmitted to the server computer 50 as thecollation image for collation. (c) of FIG. 17 shows the preview image 30in a case where the second collation is successful, and the message 44“The collation is successful.” is displayed.

Further, along with the message 44, an ID of the object 10 and thecollation date and time are displayed, and further,

a message 48 “End” is displayed.

In a case where the user selects the message 48 “end”, the processingends.

Note that in the modified example, in the first collation, light isemitted from the lower side of the ink portion 14, and in the secondcollation, the light is emitted from the left side of the ink portion14, but the light irradiation direction can be optionally changed.Although the first direction of the first collation and the seconddirection of the second collation do not necessarily have to bevertical, for example, it is suitably that the first and secondregistered images may be vertical from the viewpoint of the degree ofdeviation, the ease of understanding for the user, or the operability.Further, the first direction and the second direction on a plane may beset in accordance with the model of the collation image photographingmachine 22. For example, in a model A, the first direction on a plane isset to the lower side of the ink portion 14, the second direction on aplane is set to the left side of the ink portion 14, in a model B, thefirst direction is set to the right side of the ink portion 14, thesecond direction is set to the lower side of the ink portion 14, or thelike. The processor 22 c determines the first direction and the seconddirection for each model according to a program.

In the modified example, the collation accuracy is improved byperforming the collation twice in total, but the collation may beperformed three times or more as needed.

In the embodiments above, the term “processor” refers to hardware in abroad sense. Examples of the processor include general processors (e.g.,CPU: Central Processing Unit) and dedicated processors (e.g., GPU:Graphics Processing Unit, ASIC: Application Specific Integrated Circuit,FPGA: Field Programmable Gate Array, and programmable logic device). Inthe embodiments above, the term “processor” is broad enough to encompassone processor or plural processors in collaboration which are locatedphysically apart from each other but may work cooperatively. The orderof operations of the processor is not limited to one described in theembodiments above, and may be changed.

The foregoing description of the exemplary embodiments of the presentinvention has been provided for the purposes of illustration anddescription. It is not intended to be exhaustive or to limit theinvention to the precise forms disclosed. Obviously, many modificationsand variations will be apparent to practitioners skilled in the art. Theembodiments were chosen and described in order to best explain theprinciples of the invention and its practical applications, therebyenabling others skilled in the art to understand the invention forvarious embodiments and with the various modifications as are suited tothe particular use contemplated. It is intended that the scope of theinvention be defined by the following claims and their equivalents.

What is claimed is:
 1. A collation device comprising: a light sourceunit; a camera unit that receives light emitted from the light sourceunit and reflected in a collation area of an object to acquire aphotographed image of the collation area; a display unit; and aprocessor configured to, by executing a program: display a guide of abright spot portion of the light source unit on the photographed imageon the display unit so that an angle formed by the light source unit,the camera unit, and the collation area of the object is a predeterminedangle.
 2. A collation device comprising: a light source unit; a cameraunit that receives light emitted from the light source unit andreflected in a collation area of an object to acquire a photographedimage of the collation area; a display unit; and a processor configuredto, by executing a program: display a guide that changes in accordancewith a change in an angle of the light source unit on the display unitso that an angle formed by the light source unit, the camera unit, andthe collation area of the object is a predetermined angle.
 3. Acollation device comprising: a light source unit; a camera unit thatreceives light emitted from the light source unit and reflected in acollation area of an object to acquire a photographed image of thecollation area; a display unit; and a processor configured to, byexecuting a program: display a guide indicating a state of at least oneof a color or a pattern at a predetermined angle that changes inaccordance with a change in an angle of the light source unit on thedisplay unit so that an angle formed by the light source unit, thecamera unit, and the collation area of the object is a predeterminedangle.
 4. The collation device according to claim 1, wherein theprocessor is configured to: determine whether or not the angle formed bythe light source unit, the camera unit, and the collation area of theobject matches the predetermined angle, and display a determinationresult on the display unit.
 5. The collation device according to claim2, wherein the processor is configured to: determine whether or not theangle formed by the light source unit, the camera unit, and thecollation area of the object matches the predetermined angle, anddisplay a determination result on the display unit.
 6. The collationdevice according to claim 3, wherein the processor is configured to:determine whether or not the angle formed by the light source unit, thecamera unit, and the collation area of the object matches thepredetermined angle, and display a determination result on the displayunit.
 7. The collation device according to claim 4, wherein theprocessor is configured to: display a guide or a message that guides auser in a direction in which the angle formed by the light source unit,the camera unit, and the collation area of the object matches thepredetermined angle on the display unit, in a case where the angle doesnot match the predetermined angle.
 8. The collation device according toclaim 5, wherein the processor is configured to: display a guide or amessage that guides a user in a direction in which the angle formed bythe light source unit, the camera unit, and the collation area of theobject matches the predetermined angle on the display unit, in a casewhere the angle does not match the predetermined angle.
 9. The collationdevice according to claim 6, wherein the processor is configured to:display a guide or a message that guides a user in a direction in whichthe angle formed by the light source unit, the camera unit, and thecollation area of the object matches the predetermined angle on thedisplay unit, in a case where the angle does not match the predeterminedangle.
 10. The collation device according to claim 1, wherein theprocessor is configured to: display the guide so that a direction of thelight emitted from the light source unit on a plane is a directiondifferent for each of a plurality of collations.
 11. The collationdevice according to claim 2, wherein the processor is configured to:display the guide so that a direction of the light emitted from thelight source unit on a plane is a direction different for each of aplurality of collations.
 12. The collation device according to claim 3,wherein the processor is configured to: display the guide so that adirection of the light emitted from the light source unit on a plane isa direction different for each of a plurality of collations.
 13. Thecollation device according to claim 10, wherein the processor isconfigured to: display the guide so that the direction of the lightemitted from the light source unit on a plane is different by 90 degreesbetween a first collation and a second collation.
 14. The collationdevice according to claim 11, wherein the processor is configured to:display the guide so that the direction of the light emitted from thelight source unit on a plane is different by 90 degrees between a firstcollation and a second collation.
 15. The collation device according toclaim 12, wherein the processor is configured to: display the guide sothat the direction of the light emitted from the light source unit on aplane is different by 90 degrees between a first collation and a secondcollation.